The present invention relates to a signal processing apparatus which is adapted for processing, by spatio-temporal derivative method, signals such as image or sound signals which are derived from a measuring object and which vary spatially and temporally. The invention also is concerned with a measuring apparatus which is adapted for detecting, from the result of the signal processing performed by the signal processing apparatus, data concerning the measuring object such as kinematic data or three-dimentional data or data concerning time difference which is used for the purpose of locating the source of a sound.
In recent years, studies and researches on computer vision have made a remarkable progress and various techniques have been proposed for detecting data concerning movement or three dimensional configuration from a series of images. In particular, the flow of image formed by projection of movement of a measuring object on a screen is referred to as "optical flow" and finds a spreading use in, for example, separation of an object from the background, determination of three-dimensional construction and positioning, and so forth. The techniques for determining an optical flow has a wide use not only in the field of computer vision but also in the field of pure pattern measurement. From this point of view, the known techniques are generally sorted into two types. One of these two types is a method known as correspondence search which is executed by repeating an operation for attaining correspondece between two successive images with respect to characteristic points wich are selected with a sufficiently high degree of density or resolution. This process is a complicated and uncertain method which is generaly called as "correspondence method". Various improvements such as hierarchy search method in which the resolution of image is progressively increased in a stepped manner, and introduction of various restricting conditions.
Another type of known techniques makes use of spatio-temporal derivative method. This technique was first utilized in detection of slight movement of TV images, and approaches have been conducted for the purpose of establishing a definite formula by employing Lagrange differentiation, as well as analytical approaches.
A system is assumed here which observes, by means of a solid-state image pickup device, an object moving along a plane. The velocity field of a preselected point (x,y) is represented by (u, v). It is also assumed that only the movement constitutes the factor of temporal change in the image. When two consecutive images f.sub.1 (x,y) and f.sub.2 (x,y) are observed with a time interval .DELTA.t, these two images locally meet the following condition. EQU f.sub.2 (x,y)=f.sub.1 (x-u.DELTA.t,y-v.DELTA.t) (1)
Provided that the deviations u.DELTA.t and v.DELTA.t are small and locally approximate constants, the right side of the formula (1) can be Taylor-developed about the preselected point (x,y) as follows so that approximation is possible down to the term of first degree. EQU f.sub.1 (x-u.DELTA.t,y-v.DELTA.t)=f.sub.1 (x,y)-u.DELTA.tf.sub.1 x(x,y)-v.DELTA.tf.sub.1 y(x,y) (2)
where, f.sub.1 x(x,y) and f.sub.1 y(x,y) represent, respectively, the x- and y-partial differentiations of f.sub.1 (x,y).
By combining formulae (1) and (2), the following formula (3) is derived. EQU f.sub.1 (x,y)-f.sub.2 (x,y).apprxeq.u.DELTA.tf.sub.1 x(x,y)+v.DELTA.tf.sub.1 y(x,y) (3)
When both sides of this formula are divided by -.DELTA.t, the left side shows the time differentiation f.sub.1 t(x,y). Since f.sub.1 x(x,y) and f.sub.1 y(x,y) are differentiations of field, they can be computed immediately on the basis of the image data and, hence, can be regarded as being known values. Thus, the formula (3) can be treated as a linear equation which includes unknowns u and v and employing f.sub.1 x(x,y) and f.sub.1 y(x,y) as scalars. This is the principle of spatio-temporal derivative method. Some methods are available for determining the velocity field by making use of this principle. Examples of such methods are:
(1) A method in which the movement is limited only to unidimensional movement; and
(2) A method in which the smoothness of the velocity field is assumed and velocity field which minimizes the coarseness measure in a manner which is compatible with the condition of formula (3).
These methods, when employed in measurement, offer various advantages as follows:
(1) Plainness and high speed owing to fact that velocity distribution can be formed solely through arithmetic operation.
(2) No necessary for preparatory information concerning the object, i.e., wide adaptability and objectivity.
(3) High resolution.
(4) Less liable to be affected by pattern deformation and, hence, reduced risk for significant error, because of use of slight deviation of image.
The correspondence search type technique, however, suffers from the following disadvantage when used as a measuring method.
(1) There is a risk that a significant error is involved due to wrong correspondence.
(2) An impractically long time is required for the processing.
The first disadvantage (1) is serious and can hardly be overcome in ordinary measuring systems which, unlike the computer vision, do not employ introduction of preparatory knowledge or information.
On the other hand, the spatio-temporal derivative method tends to be affected by external noises and its application is undesirably restricted to the cases where the deviation is small.
These problems are encountered not only when the measuring object is a moving object also in the case where images of a stationary three-dimensional object are picked up by a pair of image pick-up devices disposed on both lateral sides of the object and a three-dimensional image is formed from the thus obtained pair of images. Such problems also are involved in locating the source of a sound by the time difference detection from a pair of acoustic signals obtained through a pair of microphones. The same applies also to various image processing methods such as those which employ ultrasonic wave, nuclear magnetic resonance, X-rays and so forth, as well as the optical image processing described above. For these reasons, the image processing by spatio-temporal derivative method, despite its various advantages, has not been put into practical use.